1. Field of the Invention
This invention relates to spectral imaging, and more particularly, to a method and system for obtaining spectral images of retina.
2. Description of the Related Art
Spectral images are the images in which spectral information beyond the information that is required for producing a typical color image (that is typically based on the red, green, and blue components) is provided for every point of the image or pixel. This spectral information can be related to physiological properties of an object (e.g., physiological properties of the tissue as in retina being imaged) by choosing appropriate wavelength bands. Physiological properties can be related to different pathological conditions and can be further used clinically for diagnosis and for the indication of disease development. Therefore, the spectral images are especially useful because they incorporate physiological information together with anatomical and structural information.
A specific case in which spectral imaging is applicable is spectral imaging of the retina. Spectral imaging of the retina presents a unique opportunity for direct and quantitative mapping of retinal biochemistry. For example, blood oximetry is enabled by the strong variation of the hemoglobin absorption spectra with oxygenation. This is pertinent both to research and to clinical investigation and diagnosis of retinal diseases such as diabetic retinopathy, glaucoma, and age-related macular degeneration. These diseases are the major causes of blindness in the industrial world, in which their percentage is constantly growing as the result of environmental factors and the growth of life expectancy. In order to deal with these epidemic tendencies several screening programs have been started such as the UK National Screening Program.
The principle goal of such eye screening programs is the early detection of ‘Diabetic Retinopathy,’ wherein temporal retinal images of diabetic patients are obtained and sent for evaluation. The state of the retina is visually classified, and a referral is accordingly issued, inviting the patient to a specialist or scheduling the next retinal photography.
However, the applicability of these screening programs depends on minimizing the costs that are involved. The major contribution to these costs is the employment of professional people, especially medical doctors (MDs). For this reason, the programs are based on involving MDs only when necessary. Hence, the quality of the retinal images and the level of classification become crucial.
Further, in order to support efficient and cost-effective screening, different types of digital retinal cameras have been developed (e.g., CANON's CR-DGi and CR-1, Kowa's NONMYD7, Nidek's AFC-230/210, and Topcon's NW8.) The digital retinal cameras are designed to support efficient acquisition of retinal photographs by non-professional users and with minimal requirements on pupil dilation. Similarly, computer software has also been developed to support efficient and cost-effective networking and archiving of digital retinal photographs. However, classification of the images is performed manually, which is an intensive work and is subject to errors.
The optimal exploitation of spectral imaging of the eye presents a set of challenging problems, including the poorly characterized and poorly controlled optical environment of structures within the retina to be imaged; the erratic motion of the eyeball; and the compounding effects of the optical sensitivity of the retina and the low numerical aperture of the eye. Various systems have disclosed the basic science of spectral imaging (e.g., monitoring oxygen saturation levels by spectral imaging of the eye.) However, the conventional systems provide comparatively less sensitivity and specificity due to the time required to obtain enough spectral points to support reliable calculations. In addition, in order to eliminate the effect of eye movement, the typical speed for completing the measurement must be under 0.1 second, while the conventional systems typically require up to several seconds.
The first retinal imaging oximeter based upon photographic techniques was proposed by Hickam et al. in Circulation 27, page 375 (1963). This system disclosed a modified fundus camera that images the retina at two different wavelengths, filters the image from incandescent light sources, and extracts retinal blood vessels optical density with Beer-Lambert law. Measurements with this system have lead to inaccurate results because of the Beer-Lambert Law, which strictly limits two-wavelength oximetry only to hemolyzed solutions.
Pittman and Dulling in Applied Physiology 38, page 315 (1975), showed that more accurate results of retinal oximetry can be achieved using three wavelengths instead of two. This model took into account the scattering coefficient wavelength dependence.
Three-wavelength oximetry is based on several important principles. The first of these states that light absorption by blood depends on oxygen saturation (OS) and wavelength. Second, a relationship exists between a measurable optical quantity like optical densities and the extinction coefficient of the mixture of oxygenated hemoglobin (HbO2) and deoxygenated hemoglobin (Hb) at a given OS as explained by van Assendelft in Spectrophotometry of hemoglobin derivatives (Springfield, Ill.: Thomas 1970), page 321. Finally, optical densities at two specific wavelengths can be compared to the optical density at a third specific wavelength; hemoglobin absorption values may then be calculated and be used to accurately obtain percent OS (Pittman and Duling in Applied Physiology 38, page 315 (1975)). The advantages and disadvantages of three wavelengths using existing technology have been explored by van Norren and Tiemeijer in Vision Res. 26, page 313 (1986) and by Delori and Pflibsen in Applied Optics 27, page 1113 (1988).
Three wavelength oximetry has been adapted for real-time measurements of retinal vessel OS as described by van Assendelft in Spectrophotometry of hemoglobin derivatives (Springfield, Ill.: Thomas 1970), page 321, and by Delori and Pflibsen in Applied Optics 27, page 1113 (1988). These retinal oximeters use a bright source of non-collimated light (such as a broad-spectrum halogen or arc lamps) that is filtered to provide three selected wavelengths. The light source and the filters are cooperatively selected to provide at least one isobestic wavelength (i.e., a wavelength at which hemoglobin absorption is essentially independent of OS) and at least one wavelength for which blood absorption is dependent upon OS. To probe a selected area of the retina, the light is focused on either a large caliber retinal artery or a large caliber retinal vein. The percent OS is calculated from measurements of the light reflected from either the artery (in which hemoglobin oxygenation is relatively high) or the vein (in which hemoglobin oxygenation is relatively low), and from the retinal pigment epithelium (RPE) background. However, this technique for performing retinal oximetry is complicated to control, requires precise focusing on retinal blood vessels and a complicated filtering system to produce a multi-wavelength probe. Thereby, it limits percent OS measurements to large caliber blood vessels and does not allow OS measurements to be made in the intra-retinal capillary beds.
In contrast to the above, “Full spectrum” methods (spectral methods that employ a large number of wavelengths values) have been used to record the reflectance profile versus wavelength from the ocular fundus. “Full spectrum” techniques use a high resolution imaging spectrograph to collect the spectral information from a band of tissue in a single spatial dimension. These spectrographs typically apply diffraction gratings and prisms in the spectral measurement of tunable wavelength. “Full spectrum” methods support the addition of parameters to the models that describe the spectral properties of the living (retinal) tissue, giving rise to more accurate estimates of OS in tissues outside large caliber blood vessels. Outside the large caliber vessels, the spectral signature of hemoglobin is less dominant than in the blood vessels. Examples can be found in F. C. Delori, “Reflectometry measurements of the optic disc blood volume,” in Ocular Blood Flow in Glaucoma Means, Methods and Measurements, G. N. Lambrou, E. L. Greve eds., Berkely, Calif., Kugler and Ghedini, pp. 155-163 (1989); and F. C. Delori et al., “Spectral reflectance of the human ocular fundus,” Appl. Optics, Vol. 28, pp. 1061-1077 (1989). In 1995, Schweitzer et al. [D. Schweitzer, M. Hammer, J. Kraft, E. Thamm, E. Koenigsdoerffer, and J. Strobel, “Calibration-free measurement of the oxygen saturation in retinal vessel of men,” Proc. SPIE 2393, 210-218 (1995).] built an instrument that could image the retina spectroscopically with selecting light source wavelengths from 400 nm (15.75 micro inches) to 700 nm (27.56 micro inches) in 2 nm (0.07874 micro inch) intervals; an empirical scattering model was used in their calculations.
Gil et al. disclose in U.S. Pat. No. 6,276,798 a method and apparatus for spectral bio-imaging of the retina applying Fourier Transform to recover continuous spectra from interferograms that are obtained for each pixel by a Sagnac type interferometer. The interferometer is mounted on the video output of a fundus camera. Yoneya et al. have used such a system in various clinical studies, one of which is described in Ophthalmology 109(8), page 1521 (2002). The studies have shown that the clinical applicability of the technique is limited by the long acquisition time. Subsequently, the measured data contains noise and may not be accurate due to the movements of the eye during the acquisition.
Hirohara et al. in U.S. Patent Application No. 2007/0002276 and Mihashi et al. in U.S. Patent Application Nos. 2008/0007691 and 2008/0007692 disclose a spectroscopic fundus measuring apparatus that applies a liquid crystal tunable filter in combination with a spectral characteristic correction filter in order to select the transmission wavelength in the digital imaging system that is attached to a fundus camera. The filters are disposed either in the illumination optical system or in the light receiving system, and a special method is applied in order to shorten the wavelength shifting time upon the acquisition of the spectral image. The resulting acquisition time is still in the range of seconds. A method is provided to eliminate image position changes due to eye movements and a computer program is provided to align spectral images positions almost fully automatically.
Alabboud et al. in the Proceedings of SPIE, Volume 6631, and page 66310L (2007), describe a system comprising a liquid crystal tunable filter that is integrated into the illumination system of a conventional fundus camera to enable time-sequential, random access recording of narrow-band spectral images. Image processing techniques are used to eradicate the artifacts that may be introduced by time-sequential imaging.
Kagemann et al. in Society of Photo-Optical Instrumentation Engineer (2007) have used Fourier domain Optical Coherence Tomography (OCT) data to assess retinal blood oxygen saturation in three-dimensional disk-centered retinal tissue volumes. After removing DC and low-frequency a-scan components, an OCT fundus image is created by integrating total reflectance into a single reflectance value. Thirty fringe patterns are sampled, 10 each from the edge of an artery, adjacent tissue, and the edge of a vein, respectively. A-scans are recalculated, zeroing the DC term in the power spectrum, and used for analysis. Optical density ratios (ODRs) are calculated as ODRArt=ln(Tissue855/Art855)/ln(Tissue805/Art805) and ODRVein=ln(Tissue855/Vein855)/ln(Tissue805/Vein805) with Tissue, Art, and Vein representing total a-scan reflectance at the 805- or 855-nm (33.66 microinches) centered bandwidth. A difference between arterial and venous blood saturation was shown to be detected by this technique, suggesting that retinal oximetry may possibly be added as a metabolic measurement in structural imaging devices. However, this technology is yet to be developed completely.
In summary, all “Full spectrum” systems require an acquisition time during which the eye moves relative to the optical measuring system, giving rise to spectral distortion and patient discomfort. It is shown herein that these problems are resolved by the application of snapshot spectral imaging techniques, which remove the fundamental difficulties that are associated with time-sequential techniques.
Snapshot spectral imaging systems minimize or completely waive the problem with eye movements that distort the actual spectrum of the imaged object and aim at obtaining enough spectral information in a single exposure of the imaging detectors.
Hardarson et al. in Investigative Ophthalmology & Visual Science 47/11, page 5011 (2006), have used the MultiSpec Patho-Imager (Optical Insights, Tucson, Ariz.) on the video output of a fundus camera in order to obtain four images in four different wavelength bands on a single CCD detector array in one snapshot. Their studies show relative success in estimating OS in large retinal vessels but not in the surrounding retinal tissue. They conclude that improvement can be achieved with the incorporation of correction for additional tissue optical properties, which would require image data in more wavelength bands.
Ramella-Roman et al. in Optical Society of America 16/9, page 6170 (2008), describe a multi aperture system capable of capturing six identical images of the human fundus at six different spectral bands. The system is based on lenslet array architecture. The multi-aperture system is mounted on the image output of a fundus camera to acquire spectroscopic sensitive images of the retina vessel and ultimately to calculate OS in the retina in vivo. In vivo testing on healthy volunteers was conducted and yielded results of OS similar to the one reported in the literature, with arterial OS ˜0.95 and venous OS ˜0.5. The system suffers from several drawbacks. Among those is the need of registration among the six images that fall on the single image detector of the system. This need results from the specific properties of optical set up of the system. Additionally, a focusing screen that is used in the system in order to reduce the depth of field of the incorporated lenslets reduces the light intensity that eventually reaches the image detector, thus reducing the signal-to-noise ratio of the image. Finally, observing the spectral analysis of the results presented by Ramella-Roman et al. actually shows that the number of wavelength bands for every pixel in the image is still limiting fitting of OS model with measured data.
Johnson et al. in Journal of Biomedical Optics 12(1), 014036 (January/February 2007) describe the use of computed tomographic imaging spectrometer (CTIS) to perform snapshot hyper-spectral imaging of the eye. CTIS captures both spatial and spectral information in a single frame. Its acquisition time is constrained by the exposure time of the fundus camera on which the CTIS is mounted (typically about milliseconds) and a required signal-to-noise-ratio. It is capable of acquiring a complete spatial-spectral image cube in about 3 ms from 450 to 700 nm (17.72 to 27.56 microinches) with 50 bands, eliminating motion artifacts and pixel mis-registration. There are no narrow-band filters, and nearly all collected light (about 70%) is passed to the detector at all times. The CTIS is based on diffractive grating collimated in space and which disperses the image in two dimensions. A second lens re-images the pattern onto the image detector. This produces multiple, spectrally-dispersed, images of the retina that are recorded by a focal plane array (FPA). From the captured intensity pattern, computed-tomography algorithms are used to reconstruct the scene into a “cube” of spatial (x and y) and spectral (wavelength) information. Thus, each image is not simply composed of single wavelengths; spatial and spectral information from each object pixel is multiplexed over the entire detector array. Hence, a single acquisition contains all the information required to reconstruct the spectral image cube.
Initial results of studies on human healthy subjects show a clear distinction between veins, arteries, and background. Regions within vessel capillaries agree well with the 30 to 35% oxygen saturation difference expected for healthy veins and arteries. The saturation for most of the background spatial locations in between the capillary regions shows a tendency to be within the 90 to 100% regime. This is consistent with the subjects being healthy. As the CTIS records a multiple of spectrally-dispersed images on a single FPA, which is the detector array of a fundus camera, the genuine field of view (FOV) of the host fundus camera is reduced, typically by a factor of almost three. Accordingly, the maximal FOV of the CTIS is 18 degrees, corresponding to a 50 degrees fundus camera. Additionally, complicated calibration and extensive numerical approximations are required for recovering the spectral image, each contributing its error and SNR reduction as well as long processing time. CTIS is limited by inefficient usage of both the detector array and its large number of spectral bands when only a few are required.
Alabboud et al. in the proceedings of the SPIE, Volume 6631, and page 66310L (2007), describe a snapshot spectral imaging system and technique dubbed IRIS that employs polarizing interferometery and Wollaston prism beam splitters to simultaneously replicate and spectrally filter images of the retina into multiple spectral bands onto a single detector array. The system records eight images at eight different wavelength bands on a single photo-detector.
Results of early clinical trials acquired with IRIS together with a physical model, which enables oximetry map, were reported. However, the system as described yields a small field of view and gives rise to image intensity loss upon splitting the single-band images to their appropriate locations on the image detector. Additionally, it is based on a non-compact set that does not fit existing retinal imaging systems.
Kong et al. have used a method to develop a multispectral camera to acquire spectral images in a snapshot as described in Proc. SPIE 6915, 69153K (2008). They have used a multi-wavelength narrowband filter to replace the standard Bayer color filter on monochrome CMOS sensor of a digital camera, creating in this way a miniaturized multispectral imager. The device contains a mosaic filter for four wavelengths: 540, 577, 650, and 970 nm (38.19 microinches), with the purpose of detection of erythema and bruises in persons with darkly pigmented skin. In general term, this system is disclosed in the International Patent Application PCT/US2007/087479.
In light of the above discussion, there is a need for a method and system that provides automatic classification of diabetic retinopathy. In addition, there is a need for a method and system that may significantly affect the efficiency and cost-effectiveness of screening techniques. Further, there is a need for a method and system that enables obtaining spectral images of the retina by the aforementioned non-mydriatic retinal cameras after fitting them with already modified camera-backs. Still further, such a method and system may utilize algorithms that apply the spectral information to estimate blood hemoglobin oxygen saturation in each point of the images for automatic classification of the progress of retinal vascular diseases such as diabetic retinopathy.
Accordingly, it is an object of the invention to provide an improved method and system for spectral imaging of the eye that provides spectral points (wavelength bands) to deal with the poorly characterized and poorly controlled optical environment of structures within the retina under the compounding effects of the optical sensitivity of the retina and the low numerical aperture of the eye; without registration and spectral distortion problems that are associated with time-sequential techniques because of the erratic motion of the eye ball; and without the complexity, small field of view, and intensity loss that characterize current snapshot techniques.